Modern direct-access storage devices fall into two categories: the traditional, rotating hard disk drives (HDDs) and new solid state storage devices (SSDs). While they are presented identically to the OS, they have very different performance characteristics and a magnitude of difference in the cost per gigabyte.
Storage system vendors tend to rely on either rotating or solid state media, creating two corresponding categories of storage systems: HDD-based systems that are characterized by lower performance and lower cost, and SSD-based systems that have significantly higher performance and cost as measured by generally available performance tools.
Most unsophisticated “brute force” performance testing tools create artificial workloads, such as randomly writing 4K blocks across the entire capacity of a storage volume or a file. Although the workload is valid, it hardly represents the behavior of a real-world application.
Real-world computer data relevant to a particular processing task (or application) tend to occur in quasi stable groups or clusters. This phenomenon is called “locality of reference”, and it can typically take two forms: spatial and temporal localities. Spatial locality means that the application data is placed closely to one other, such as in a single file, folder or a storage device. Temporal locality is the notion that most of the application data will be accessed within a certain timeframe, such as when the application is being used.
Spatial and temporal localities are often correlated. In other words, an application tends to access a limited data set during a given period of time. The footprint of the data subset representing locality of reference varies depending on the applications and their behavior.
The computer industry broadly relies on the locality of reference phenomenon by deploying various techniques where a subset of active data is placed in a “cache”, or a high-speed temporary storage, and stays there until it is evicted by a higher activity workload.
The caches are a standard fixture in modern CPUs where the instructions and data are loaded (“prefetched”) from DRAM into high-speed on-chip memory on the first read access and are temporarily stored there upon writes with a subsequent write-back to the DRAM. There is usually more than one cache level, with the highest speed cache being the smallest in capacity and vice versa.
Caching technology is also used in storage systems, though to a more limited extent. Traditional storage caching solutions are based on some type of NVRAM (non-volatile SRAM or DRAM) that is relatively small (512 MB-8 GB), proprietary and expensive. Due to its limited size, the cache can hold only a small fraction (typically less than 0.1%) of storage system data, so the data is frequently and aggressively evicted. Consequently, the use of caching technology in storage systems is generally limited to addressing goals such as guaranteeing data consistency in parity RAID stripes across power loss, coalescing small sequential writes into larger ones to improve write latency, and read-ahead for sequential streams (to load transfer pipeline).
What is needed is a mechanism for efficient handling of storage I/O based on storage devices with mixed types.